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Abstract 

MIT  SchMUSE  (pronounced  “shmooz”)  is  a  concurrent,  distributed,  delegation-based  object-oriented 
interactive  environment  with  persistent  storage.  It  is  designed  to  run  in  a  “capricious”  network  envi¬ 
ronment,  where  servers  can  migrate  from  site  to  site  and  can  regularly  become  unavailable.  Our  design 
introduces  a  new  form  of  unique  identifiers  called  globally  unique  tickets  that  provide  globally  unique 
time/space  stamps  for  objects  and  classes  without  being  location  specific.  Object  location  is  achieved  by 
a  distributed  hierarchical  lazy  lookup  mechanism  that  we  call  realm  resolution.  We  also  introduce  a  novel 
mechanism  called  message  deferral  for  enhanced  reliability  in  the  face  of  remote  delegation.  We  conclude 
with  a  comparison  to  related  work  and  a  projection  of  future  work  on  MIT  SchMUSE. 
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1  Introduction 

MIT  SchMUSE  is  a  ScHEME-based  [Clinger  k  Rees  91] 
[IEEE  91]  Multi-User  Simulation  Environment  based 
on  a  concurrent,  distributed,  delegation-based  object- 
oriented  language  with  a  persistent  object  store. 
It  was  originally  conceived  as  a  teaching  vehicle 
for  the  introductory  programming  course  at  MIT 
[Abelson  k  Sussman  85]. 

The  project’s  main  goal  was  to  provide  a  SCHEME- 
based  concrete  introductory  case-study  of  computer  sys¬ 
tems  issues  such  as  concurrency,  distributed  comput¬ 
ing,  persistence,  recoverability,  transactions,  reliability, 
and  delegation-based  object-oriented  programming  sys¬ 
tem  (OOPS).  We  did  so  under  the  guise  of  implementing 
a  multi-user  interactive  adventure  game,  mainly  because 
that  is  a  common  setting  very  familiar  to  most  students. 
It  was  not  our  intention,  however,  to  limit  ourselves 
to  this  application  domain.  For  example,  we  envision 
the  SchMUSE  as  being  a  practical  prototype  system 
in  which  to  implement  a  distributed  interactive  virtual 
office  space  or  a  distributed  object-oriented  database. 
Nonetheless,  for  reader  accessibility,  the  examples  in  this 
report  reflect  a  simulated  world  setting.  Moreover,  this 
report  focuses  exclusively  on  the  novel  features  of  the 
SchMUSE  language  design  and  its  functionality  rather 
than  on  distributed  systems  issues  per  se.  To  wit,  we 
view  our  primary  contribution  as  having  explored  several 
novel  language  systems  issues  that  arise  in  distributed 
object  delegation  with  regards  to  language  implementa¬ 
tion  rather  than  in  having  tackled  the  distributed  sys¬ 
tems  issues  that  arise  merely  by  virtue  of  being  a  dis¬ 
tributed  computing  environment,  such  as  message  trans¬ 
action  atomicity  and  concurrency  coordination. 

We  chose  to  implement  our  system  in  MIT  Scheme 
[Hanson  91],  a  dialect  of  the  Lisp  family  of  program¬ 
ming  languages.  Our  object  system  is  modelled  after  the 
delegation-based  OOPS  style  of  [Adams  k  Rees  88],  pri¬ 
marily  because  that  is  the  language  in  which  our  course 
is  taught.  None  of  our  results  are  specific  to  Scheme; 
we  could  equally  as  well  have  implemented  this  system 
in  Smalltalk-80  [XLRG  81],  CLOS  [Bobrow  et  al  90], 
Dylan  [Apple  92]  or  even  C++  [Stroustrup  86].^ 

1.1  Application  Context 

The  laboratory  setting  in  which  we  envision  MIT 
SchMUSE  being  exercised  has  greatly  influenced  its  de¬ 
sign.  Specifically,  our  course  lab  is  comprised  of  a  local 
area  network  of  46  high-performance  diskless  worksta¬ 
tions  (HP  720s)  evenly  divided  across  2  high-powered 
centralized  network  file  servers  (HP  750s).  Ultimately, 
however,  we  envision  MIT  SchMUSE  spreading  across 
the  Internet  as  an  extremely  distributed  global  network 

^Of  course,  delegation-based  dialects  of  these  lan¬ 
guages  would  be  required.  The  exemplar-based  dialect  of 
Smalltalk  [LaLonde  86]  could  be  used.  Delegation-based 
extensions  can  be  made  fairly  painlessly  to  CLOS  and  Dy¬ 
lan  by  careful  use  of  the  Meta-Object  Protocol  (MOP) 
[Kiczales  et  al.  91],  as  demonstrated  in  [Blair  k  Maessen  93]. 
Finally,  the  feasibility  of  delegation  in  C++  was  demonstrated 
in  [Johnson  k  Zweig  91]. 


of  potentially  collaborating  simulation  sites.  For  exam¬ 
ple,  now  that  MIT  provides  network  access  in  the  cam¬ 
pus  dormitories,  we  would  not  be  surprised  if  our  stu¬ 
dents  choose  to  leave  their  otherwise  idle  PCs  active  as 
SchMUSE  servers,  running  atop  publicly  available  DOS 
MIT  Scheme  [DOS  Scheme]. 

The  classroom  mode  of  operation  for  students 
ScnMUSE-ing  is  envisioned  to  be  as  follows:  a  stu¬ 
dent  logs  into  an  unused  workstation  and  brings  up  a 
local  SchMUSE  client  session.  During  this  session, 
the  student  can  interact  (via  TCP/IP  sockets)  with  ob¬ 
jects  on  either  of  two  highly  available  centralized  file 
servers,  likely  creating  a  few  new  instances  of  standard¬ 
ized  classes  on  the  servers,  subject  to  a  modest  space 
quota. 

More  interestingly,  the  student  can  also  extend  the 
simulation  system  with  new  classes  and  objects  on  their 
client  machine.  These  new  creations  can  then  be  made 
available  to  other  students  in  the  lab  by  allowing  other 
clients  in  the  lab  to  establish  server  connections  to  their 
client.  In  this  way,  we  blur  the  distinction  between  server 
and  client  since  we  allow  clients  to  themselves  act  as 
servers  to  other  clients.  This  permits  very  interactive 
collaboration  throughout  the  network.  A  student  on  one 
workstation  can  build  on  the  efforts  of  students  on  sev¬ 
eral  other  workstations,  and  vice  versa,  to  build  elabo¬ 
rate  team  experiments. 

When  a  student  logs  out  of  a  workstation,  unfortu¬ 
nately  their  client/server  sessions  must  be  terminated 
(to  make  room  for  the  next  student),  but  their  state 
can  be  dumped  to  persistent  storage.  Specifically,  their 
code,  as  well  as  the  state  of  their  local  object  instance 
database,  can  be  dumped  as  files  to  a  personal  floppy 
disk  or  f  tp’d  to  a  remote  file  server. 

When  the  student  returns  to  the  lab  (or  to  their  dorm 
room),  all  their  SchMUSE  state  can  be  restored  from 
their  last  state  dump  and  their  session  can  continue.  Of 
course,  some  of  the  other  student  servers  they  were  in¬ 
teracting  with  may  have  also  moved  to  a  different  work¬ 
station  {e.g.,  so  the  student  can  sit  closer  to  someone  else 
they  are  collaborating  with  or  because  their  reservation 
on  the  machine  they  were  using  had  expired)  or  they  may 
be  altogether  unavailable  [e.g.,  the  student  who  was  run¬ 
ning  the  server  session  may  be  temporarily  in  transit  to 
a  new  location  or  dormant  during  sleeping  and  eating 
periods). 

1.2  Outline 

The  following  sections  deal  with  the  difficulties  of  mit¬ 
igating  this  sort  of  capricious  network  environment  of 
itinerant  virtual  servers,  where  servers  can  become  un¬ 
available  and  where  servers  can  move  among  worksta¬ 
tions,  both  with  disturbing  regularity.  First,  however, 
we  establish  our  language  context  by  describing  the 
SchMUSE  object  system  in  section  2.  In  section  3,  we 
proceed  to  describe  how  remote  delegation  is  supported 
in  our  object  system.  In  section  4,  we  show  how  impos¬ 
ing  class  structure  on  our  delegation  system  enables  var¬ 
ious  performance  enhancements  for  remote  delegation. 
Section  5  introduces  message  deferral  for  improved  reli¬ 
ability  in  a  system  where  servers  are  itinerant.  Section  6 


relates  our  language  design  to  other  work  in  the  field. 
Section  7  emphasizes  the  unique  contributions  of  this 
work.  Finally,  in  section  8,  we  conclude  by  outlining 
future  work  in  the  project. 

2  The  SchMUSE  Object  System 

The  sole  means  of  object  interaction  in  MIT  SchMUSE 
is  through  message  passing  [Agha  86].  Methods  are  in¬ 
voked  by  passing  messages  to  objects,  and  instance  slots 
are  accessed  via  messages.  (It  will  become  clear  below 
why  we  choose  to  distinguish  object  slot  accesses  from 
other,  more  compound  methods). 

Unlike  prototype-based  delegation  systems 
[Lieberman  86],  our  system  employs  class-based  delega¬ 
tion.  Specifically,  our  objects  are  created  as  instances 
of  classes,  where  each  class  declares  the  local  variables 
and  methods  defined  on  instances  of  the  class  as  well 
as  declaring  any  parent’s  classes.^  When  an  instance  of 
a  class  is  generated,  a  chain  of  partial  objects  is  made 
which  corresponds  to  the  inheritance  chain  prescribed 
by  the  object’s  class  structure.  An  example  is  sketched 
in  Figure  1.  Each  partial  object  (box),  hereafter  called 
a  node,  contains  slots  for  the  local  variables  as  well  as 
pointers  to  node  instances  of  the  node’s  parent  classes. 
Notice  that  a  node  taken  with  the  transitive  closure  of 
its  parent  nodes  constitutes  an  object  instance.  A  node 
is  said  to  delegate  to  its  parent  objects.  In  this  way,  an 
object  is  represented  as  a  directed  acyclic  graph  (DAG) 
of  nodes  that  directly  refiects  the  inheritance  structure 
of  its  class  definition. 

Note  from  Figure  1  that  slot  names,  in  addition  to 
method  names,  can  shadow  one  another.  For  example, 
persons  and  students  both  have  a  nickname  slot.  In  ad¬ 
dition,  as  our  figure  suggests,  an  object  can  be  the  parent 
of  more  than  one  child  node.  In  our  figure,  the  person 
is  a  common  parent  of  both  a  student  node  and  an  em¬ 
ployee  node. 

This  very  general  framework  for  sharing  is  what  makes 
delegation-based  inheritance  most  compelling  for  dis¬ 
tributed  systems.  Specifically,  in  a  distributed  setting, 
each  object  node  could  reside  on  a  separate  workstation. 
We  call  this  distribution  of  delegate  nodes  across  several 
sites  remote  delegation.  Note  that  traditional  class-based 
inheritance  systems  such  as  Smalltalk  [XLRG  81], 
CLOS  [Bobrow  et  al  90],  and  C++  [Stroustrup  86]  pro¬ 
vide  no  such  distributed  sharing.  Instead,  they  flat¬ 
ten  objects  into  one  long  array  of  slots,  one  per  each 
unique  slot  name.  Consequently,  slot  shadowing  is  not 
supported  in  these  systems.  Also,  slot  sharing  among 
multiple  children  is  not  directly  supported.^ 

Delegation-derived  sharing  allows  for  complex  pat¬ 
terns  of  centralized  sharing  with  privacy  control.  In  our 
example,  for  instance,  centralized  sharing  is  illustrated 

^We  permit  multiple  inheritance.  For  simplicity,  we  fol¬ 
low  the  mechanism  of  left-to-right  topological  linearization 
of  multiple  parents  described  in  [Snyder  86]  and  used  as  the 
default  in  CLOS  [Bobrow  et  al.  90]  and  Dylan  [Apple  92]. 

^Sorne  contortions  involving  indirection  through  shared 
cell  data  is  possible.  This  is  what  we  resorted  to  in 
[Blair  k  Maessen  93]. 


by  the  person  being  shared  by  both  the  student  and  em¬ 
ployee  nodes.  By  centralizing  the  sharing,  privacy  can 
be  enhanced;  for  instance,  the  employee  child  may  be 
granted  access  to  the  person’s  social  security  number 
while  the  student  child  may  be  denied  this  information. 
Moreover,  by  permitting  multiple  independent  children 
we  in  effect  provide  multiple  independent  views  of  the 
same  essential  object.  Specifically,  an  application  that 
knows  about  the  student  view  of  our  person  may  be  un¬ 
aware  of  our  employee  view.  Thus,  a  student’s  advisor 
may  be  unaware  that  s/he  is  consulting  on  the  side.  This 
too  is  an  important  form  of  privacy. 

3  Remote  Delegation  via  Globally 

Unique  Tickets  and  Realm  Resolution 

All  objects  in  the  SchMUSE  are  passed  to  methods 
via  object  reference.  In  order  to  support  remote  del¬ 
egation,  these  object  references  must  embody  globally 
unique  identifiers  [Leach  et  al.  82]. 

3.1  Globally  Unique  Tickets 

In  MIT  SchMUSE  this  need  for  globally  unique  IDs  for 
object  references  is  accomplished  by  implementing  ev¬ 
ery  object  reference  as  an  object  ticket.  Similarly,  class 
references  are  implemented  as  class  tickets.  Collectively, 
object  and  class  tickets  comprise  what  we  call  globally 
unique  tickets.  The  grammar  for  these  GUTs  of  the 
SchMUSE  object  system  is  shown  in  Figure  2. 

The  first  field  of  each  ticket  is  not  strictly  necessary: 
they  are  introduced  primarily  for  debugging  support,  al¬ 
though  we  will  later  exploit  the  ClassTicket  within  an 
ObjectTicket.  A  btime  is  the  encoding  of  a  millisecond 
real-time  clock  reading  of  when  the  entity  was  born,  and 
bmachloc  is  the  encoding  of  the  “birth  machine  loca¬ 
tion”,  i.e.,  the  network  address  of  the  machine  on  which 
the  entity  was  created.  Together,  these  two  data  pro¬ 
vide  a  globally  unique  identifier  for  every  object  in  the 
network.^  They  do  not,  however,  provide  location  infor¬ 
mation  if  we  allow  SchMUSE  sessions  to  migrate  about 
the  network.  This  is  where  the  RealmTicket  comes  into 
play. 

Each  server /client  session  maintains  its  own  names¬ 
pace  of  classes  and  instances  in  the  underlying  Scheme 
session.  A  class  table  and  object  table  are  maintained  to 
map  class  tickets  to  classes  and  object  tickets  to  object 
instances.  Since  SchMUSE  sessions  are  itinerant,  these 
namespaces  are  consequently  mobile.  We  conceptually 
divide  each  namespace  into  realms  of  course-grained  col¬ 
lections  of  objects.  For  example,  a  student  may  be  devel¬ 
oping  an  adventure  game  simulation  with  a  Blade  Run¬ 
ner  Realm  and  a  Time  Bandits  Realm.  Each  separate 
realm  can  be  dumped  to  disk  and  transported  to  another 
workstation  separately. 

When  an  object  class  or  instance  is  created,  it  is  al¬ 
ways  created  in  some  realm.  This  realm  information  is 
encoded  by  the  RealmTicket  inside  the  entity’s  ticket. 

^Note  also  that  they  obviate  the  need  for  distributed  clock 
synchronization:  so  long  as  each  workstation  assures  that  its 
own  clock  processes  strictly  forward,  no  two  distinct  entity 
tickets  can  collide  in  SchMUSE  space-time. 
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Figure  1:  An  example  delegation-based  instance. 
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RealmTicket  ::=  realmname  x  btime  x  bmachloc 

Figure  2:  Globally  Unique  Tickets 


When  remote  sessions  access  a  local  realm  and  receive 
copies  of  its  local  object  tickets,  this  realm  information 
is  later  used  to  locate  the  object  again.  That  is,  when  a 
message  is  sent  to  an  object  ticket,  if  the  object  ticket’s 
realm  matches  the  realm  we  are  in  when  processing  the 
message,  then  the  object  must  be  local.  The  local  object 
table  is  then  used  to  map  the  object  ticket  to  the  desired 
object  and  the  message  can  be  processed  on  that  local 
object.  If,  on  the  other  hand,  the  object  ticket’s  realm  is 
not  that  of  a  local  realm,  then  the  message  is  forwarded 
to  the  machine  where  the  remote  object’s  realm  resides. 

3.2  Realm  Resolution 

Notice,  however,  that  RealmTicket s  pointedly  do  not 
contain  location  information  other  than  the  birthplace 
of  a  realm.  Thus,  there  is  nothing  in  the  ticket  to  de¬ 
clare  where  a  particular  realm  actually  resides  in  the  net¬ 
work.  For  this  we  resort  to  a  “Brazilian”  ^  distributed 
hierarchical  realm  resolution/location  mechanism  which 
we  have  dubbed  Central  Services.  Central  Services  is 
not  a  single  centralized  entity  but,  rather,  a  distributed 
hierarchical  network.  Each  SchMUSE  session  which  ini¬ 
tiates  server  access  does  so  by  contacting  some  nearby 
authority  and  requesting  access  to  Central  Services.  The 
authority  designates  some  server  within  the  network  of 

®That  is,  inspired  by  the  movie  Brazil 


active  SchMUSE  servers  to  serve  as  realm  resolution 
authority  for  the  new  session.  In  this  way,  a  variety  of 
load-distributed  resolution  hierarchies  can  be  initiated 
by  the  central  authority,  such  as  N-ary  trees  and  H-trees 
[Leighton  92]. 

The  server  then  informs  the  authority  and  its  desig¬ 
nated  Central  Services  contact  point  of  the  realms  which 
it  is  making  available  for  server  access.  This  information 
is  then  lazily  distributed  throughout  the  SchMUSE  net¬ 
work  on  demand  in  a  fashion  similar  to  Internet  names¬ 
pace  service.  Should  this  contact  point  become  unavail¬ 
able,  a  new  one  can  be  requested  from  the  nearby  author¬ 
ity.  The  authority  network,  being  the  backbone  of  the 
SchMUSE,  is  fixed  and  distributed,  much  like  Internet 
namespace  service. 

Each  SchMUSE  session,  then,  maintains  a  cache  of 
its  realm  lookups  to  map  RealmTickets  to  real  machine 
addresses.  Of  course,  these  address  mappings  will  be¬ 
come  obsolete  when  a  remote  server  session  terminates, 
so  a  time-out  mechanism  is  implemented  whereby  an  un¬ 
successful  connection  can  resort  to  querying  Central  Ser¬ 
vices  to  see  if  the  desired  realm  has  moved.  When  a 
session  is  terminated,  it  is  customary  to  therefore  no¬ 
tify  Central  Services  so  that  future  inquires  regarding 
the  realm’s  location  can  be  rejected  without  resorting  to 
repeated  time-out  mechanisms. 


The  details  of  this  distributed  hierarchical  realm  res¬ 
olution  mechanism  of  the  SchMUSE  are  still  very  much 
in  the  experimental  phase  so  little  can  be  said  of  its  per¬ 
formance.  Nonetheless,  we  are  confident  that  this  style  of 
lazy  location  with  eager  cancellation  will  suit  our  labora¬ 
tory  and  campus  environment  well.  We  have  also  found 
that  performance  profiling  and  system  debugging  were 
greatly  facilitated  by  separating  an  entity’s  birth  ma¬ 
chine  location  from  its  present  location  hint  associated 
the  object  tickets.  Specifically,  the  btime  and  bmachloc 
remain  constant  despite  realm  motion,  providing  a  time- 
invariant  UID  by  which  to  identify  objects  in  the  net¬ 
work. 

4  Using  Class  Information  for  Efficient 
Remote  Delegation 

We  turn  now  to  the  performance  issue  of  making 
delegation-based  inheritance  efficient  in  the  face  of  re¬ 
mote  delegation.  In  short,  we  use  static  information  from 
the  class  definitions  to  accelerate  method  dispatch  and 
slot  accesses.  Most  importantly,  the  static  class  infor¬ 
mation  allows  us  to  delegate  directly  to  the  node  where 
a  slot  is  located  rather  than  traversing  the  full  delega¬ 
tion  chain  of  the  object  being  manipulated.  In  the  case 
of  highly  distributed  delegation,  this  can  have  major 
performance  improvements  since  superfluous  delegation 
chain  traversal  amounts  to  superfluous  network  traffic 
and  unnecessary  indirection  through  numerous  worksta¬ 
tions,  tying  up  valuable  resources  and  heightening  the 
liability  of  access  failure.  In  the  case  where  the  com¬ 
plete  structure  information  about  a  node  is  not  present 
on  the  SchMUSE  session  that  is  processing  a  message, 
we  must  be  a  bit  more  clever. 

In  brief,  the  performance  issues  surrounding  remote 
delegation  can  be  divided  along  two  lines  of  concern: 
first,  the  amount  of  structure  information  available  con¬ 
cerning  the  object  being  manipulated;  and  second,  the 
kind  of  message  being  processed  by  the  object.  These 
concerns  naturally  subdivide,  in  turn,  as  follows:  regard¬ 
ing  structure  information,  we  must  consider  1)  if  the  full 
class  information  is  co-resident  with  the  instance,  or  2) 
if  we  allow  opaque  remote  classes,  we  can  have 

a  local  instance  whose  local  class  information  may  be 
known  but  whose  inherited  class  structure  is  remote  and 
not  known.  As  we  shall  see  shortly,  these  two  class  poli¬ 
cies  have  different  implications  depending  on  whether 
the  message  involved  is  A)  a  slot  access  message,  or  B)  a 
compound  method  message.®  The  following  subsections 
address  each  of  the  four  resulting  combinations. 

4.1  Case  lA:  Full  class  info;  Slot  access 

Returning  to  our  example  delegation  in  Figure  1,  imag¬ 
ine  we  are  on  the  machine  where  the  student  node  re¬ 
sides  and  we  wish  to  access  the  student’s  place  slot. 
Imagine  that  each  ancestral  delegate  node  resides  on  a 
separate  machine.  In  such  a  configuration,  if  we  were 
naive  in  accessing  the  place  slot  of  the  student,  we  would 
have  to  traverse  the  object’s  delegation  DAG  through  the 

®That  is,  a  method  whose  body  is  a  mixture  of  slot  refer¬ 
ences  and  Scheme  procedure  calls. 


Person  node  to  get  to  the  MobileObject  node  where  the 
place  slot  resides.  If,  however,  we  locally  knew  the  ob¬ 
ject  tickets  in  the  full  object  class  ancestry  of  the  student 
and  we  further  knew  that  the  place  slot  was  accessed 
as  slot  0  of  the  student’s  parent’s  primary  parent  (en¬ 
coded  as  delegate  <0,0>),  then  we  could  directly  send  to 
the  MobileObject  node  a  request  for  the  value  of  slot 
0.  This  would  entirely  bypass  the  Person  machine  and 
would  lighten  the  burden  on  the  MobileObject  machine 
by  not  engaging  it  to  decode  the  place  message  into  a 
slot  access  on  slot  0. 

We  implemented  such  a  strategy  as  follows:  at  class 
creation  time,  we  “compile”  the  slot  access  methods  to 
operate  in  terms  of  “lexical  addresses”  into  the  delega¬ 
tion  chain.  Specifically,  at  method  installation  time,  we 
statically  compute,  for  each  slot  accessor  method,  both 
the  offset  into  the  delegation  chain  for  the  delegate  which 
possesses  the  target  slot  and  the  index  of  that  slot  within 
the  target  delegate’s  array  of  slots.  We  further  arrange 
that  each  local  node,  upon  creation,  cache  away  a  DAG 
representing  its  full  delegation  ancestry.  In  this  way, 
whenever  a  slot  access  method  is  called  on  a  local  node, 
the  encoded  indirection  into  the  delegation  chain  can 
be  used  to  indirect  into  the  cached  delegation  ancestry 
DAG.  The  encoded  slot  access  can  then  be  sent  directly 
to  the  object  ticket  representing  the  target  delegate.  In 
our  prototype  implementation,  we  witnessed  an  order  of 
magnitude  in  performance  improvement  when  this  en¬ 
hancement  was  installed:  a  test  case  that  took  20  min¬ 
utes  to  complete  without  it  took  only  2-3  minutes  with 
it.*^ 

4.2  Case  IB:  Pull  class  info;  Compound 
method 

Compound  methods  can  likewise  be  decomposed  into 
more  primitive  operations.  Specifically,  every  method 
ultimately  decomposes  into  nested  sequences  of  Scheme 
language  primitives  and  slot  accesses.  Thus,  if  we  can 
know  the  implementation  of  the  Animat eObject’s  hear 
method  locally  on  the  machine  where  the  Student  node 
inherits  it,  then  hear  messages  sent  to  the  Student  can 
be  decomposed  on  the  local  Student  node’s  machine  into 
Scheme  primitives  and  slot  accesses.  This  means  that 
the  bulk  of  the  work  involved  in  processing  a  hear  mes¬ 
sage  can  be  assumed  by  the  machine  on  which  the  mes¬ 
sage  was  initiated.  This  dramatically  reduces  the  load 
on  the  remote  machine  where  the  AnimateObject  node 
resides,  enhancing  its  availability  to  other  remote  child 
nodes.  In  our  prototype  implementation,  we  witnessed, 
on  average,  a  4-5  fold  decrease  in  load  on  the  remote 
node  when  this  feature  was  enabled. 

4.3  Cases  2 A  &  2B;  Remote  class  info;  Slot  or 
Method  messages 

The  natural  question  to  follow  from  the  preceding  is,  of 
course,  what  to  do  if  the  full  class  information  (namely, 
delegate  storage  and  inherited  methods)  is  not  available 
to  some  remote  child.  For  example,  imagine  that  the 

’^This  test  case  involved  an  obscenely  complex  remote  del¬ 
egation  DAG  being  pounded  on  during  peek  network  usage 
hours. 


local  slot  structure  and  local  methods  for  a  Student  are 
known  at  the  student’s  node  but  that  the  cletss  informa¬ 
tion  for  a  Person  is  not  known.  This  may  be  a  reasonable 
situation  if,  say,  the  Person  class  implementor  has  de¬ 
cided  not  to  export  the  implementation  of  persons.®  In 
this  case,  any  message  that  we  can  decode  locally  using 
only  information  known  about  local  students  can  still 
be  done  with  some  efficiency,  yet  any  inherited  message 
would  be  an  unknown. 

In  such  a  case,  we  re-package  the  message  and  forward 
it  to  the  machine  where  the  remote  class  resides,  us¬ 
ing  the  RealmTicket  in  the  ClassTicket  of  the  object’s 
parent  to  determine  its  residence.®  In  this  re-pac^ged 
message  we  are  also  careful  to  include  the  object  ticket 
of  the  Student  object  for  which  the  message  was  origi¬ 
nally  intended.  It  is  then  the  responsibility  of  the  remote 
class  to  ultimately  discover  the  method  for  this  mes¬ 
sage.  This  may,  in  turn,  involve  further  re-packaging  to 
other  remote  classes.  Nonetheless,  assuming  an  applica¬ 
ble  method  is  discovered,  the  invocation  of  this  method 
on  the  intended  object  ticket  will  naturally  result  in  the 
appropriate  decomposition  of  the  message  into  Scheme 
primitives  and  slot  accesses.  Thus,  the  price  for  keeping 
a  class  implementation  private  is  the  cost  of  performing 
all  non-exported  methods  at  the  class  site.  Notice  that 
this  suggests  that  some  methods  may  be  exported  and 
some  not,  depending  on  their  depth  within  the  inher¬ 
itance  structure.  Notice,  further,  that  in  the  extreme 
case  where  no  classes  are  exported,  the  remote  class  re¬ 
packaging  strategy  will  ultimately  result  in  a  full  traver¬ 
sal  of  the  delegation  structure.  This  ultimately  degrades 
into  the  naive  delegation  strategy  outlined  at  the  begin¬ 
ning  of  case  lA. 

If  the  unknown  re-packaged  message  is  found  to  cor¬ 
respond  to  a  simple  slot  access  method,  it  is  then  de¬ 
sirable  to  request  the  intended  object  ticket  to  reveal 
its  full  ancestry  so  that  the  slot  access  optimization  can 
proceed  by  using  this  ancestry  DAG  to  avoid  travers¬ 
ing  the  delegation  chain,  analogous  to  case  lA  above.^® 
This  is  very  helpful  in  the  case  where  some  intermediate 
ancestor  is  only  partially  opaque  (like  Person)  but  some 
deeper  ancestor  atop  which  it  is  built  is  exported  (like 
MobileObject). 

If,  on  the  other  hand,  the  message  is  discovered  to  be  a 

®For  example,  this  class  may  be  still  under  construction 
or  one  of  its  methods  may  use  proprietary  code  or  code  under 
federal  export  control. 

®This,  of  course,  is  complicated  by  multiple  inheritance. 
In  such  a  situation,  each  remote  parent  class  would  have  to 
be  attempted,  in  turn,  until  one  of  them  is  able  to  process 
the  message,  after  which  the  deferral  target  can  be  cached  to 
avoid  repeated  exhaustive  searches. 

^°The  reason  this  is  only  “analogous”  to  case  lA  is  that  a 
trick  is  involved.  The  trick  is  to  notice  that  in  re-packaging 
to  some  delegate  class,  the  offset  into  the  delegation  chain 
that  that  class’s  methods  will  embody  will  not  include  the 
delegation  through  the  ancestry  of  the  intended  object  up 
to  that  remote  class.  This  is  easily  handled  by  the  remote 
class  by  first  walking  through  the  delegation  ancestry  of  the 
intended  object  to  get  to  the  delegate  ancestor  corresponding 
to  the  remote  class.  Once  there,  it  can  proceed  by  applying 
the  slot  access  method  to  that  object  ticket. 


compound  method,  then,  similar  to  case  IB,  the  method 
can  be  decomposed  into  more  primitive  operations  and 
processing  proceeds  as  sketched  above.  Notice  that  it  is 
not  until  we  tackle  the  message  at  the  remote  class’s  host 
that  we  discover  whether  the  message  is  a  slot  access  or 
a  compound  method. 

4.4  Summary  of  Efficient  Remote  Delegation 

In  closing  on  this  issue  of  efficient  remote  delegation  in 
the  face  of  full/remote  classes  a  few  points  should  be 
emphasized. 

First  of  all,  it  should  be  noted  that  we  intend  to  re¬ 
quire  every  local  object  to  at  least  have  local  class  infor¬ 
mation  on  the  machine  where  it  resides.  Specifically,  we 
consider  it  undesirable  to  have  a  local  Student  object 
instance,  say,  on  a  machine  that  does  not  know  about 
the  Student  class.  Such  a  local  object  would  be  use¬ 
less  in  that  every  message  to  it  would  ultimately  have 
to  be  forwarded  somewhere  else  only  to  have  the  most 
primitive  slot  accesses  actually  performed  on  the  object 
locally. 

Second,  we  do  not  have  a  problem  with  several  inde¬ 
pendent  implementations  of  the  “same”  class  {e.g.,  two 
distinct  kinds  of  Student)  since  each  class  is  stamped 
by  when/where  its  implementation  was  born.  Specif¬ 
ically,  when  a  class  implementation  is  exported,  the 
ClassTicket  for  the  class  at  the  exporting  site  is  em¬ 
bedded  within  the  exported  class  code.  Thus,  distinct 
re-implementations  of  a  class  (i.e.,  version  updates)  will 
have  different  ClassTickets  since  their  timestamps  (at 
least)  will  differ.  Similarly,  exporting  a  class  to  a  ma¬ 
chine  that  then  extends  or  otherwise  modifies  the  class 
will  again  be  reflected  in  a  new  distinct  ticket  for  the 
resulting  new  class.  To  our  knowledge,  no  other  system 
has  adopted  such  a  clean  approach.  The  use  of  class  ver¬ 
sion  numbers  comes  close,  but  requires  global  synchro¬ 
nization  on  the  issuance  of  these  version  numbers.  Our 
technique  requires  no  such  synchronization.  We  find  the 
globally  unique  ticket  mechanism  to  be  quite  elegant  in 
this  regard. 

Finally,  it  is  worth  re-iterating  that  at  any  point  in 
the  class  hierarchy  one  might  choose  to  export  or  not 
export  an  ancestral  class  implementation.  Thus,  hierar¬ 
chical  privacy  and  opaque  encapsulation  are  supported  in 
the  SchMUSE.  This  mechanism,  in  effect,  supports  ab¬ 
stract  types  in  a  distributed  object-oriented  framework. 
The  burden,  of  course,  is  that  the  site  of  the  implemen¬ 
tation  is  charged  with  processing  the  messages  to  such 
objects.  We  have  not  explored  the  ramifications  of  this 
policy  in  detail,  but  we  consider  it  to  be  intriguing  that 
to  keep  an  implementation  aspect  private  or  proprietary 
one  must  pay  the  overhead  of  servicing  requests  for  these 
secretive  aspects  oneself  or  arrange  to  have  them  serviced 
only  by  trusted  authorized  or  licensed  servers. 

5  Message  Deferral  for  Reliable 
Remote  Delegation 

We  now  move  from  the  efficiency  issues  surrounding  re¬ 
mote  delegation  to  the  reliability  issues.  Specifically,  a 
distributed  system  is  reliable  if  its  performance  and  avail- 


ability  are  not  affected  by  machines  crashing.  We  have 
already  seen  how  delegating  directly  to  nodes  that  con¬ 
tain  slots  allows  us  to  jump  over  intermediate  delegate 
nodes.  This  certainly  enhances  reliability  in  the  case 
where  the  jumped  node  may  not  have  been  up. 

Otherwise,  if  some  delegate  is  unavailable  in  our 
system  due,  for  example,  to  the  server  that  supports 
that  delegate  not  being  currently  “plugged  into  the 
SchMUSE”  then  we  can  just  avoid  any  slot  accesses 
to  slots  local  to  that  delegate  and  hence  avoid  thrash¬ 
ing  the  Central  Services  in  search  of  a  realm  that  is  not 
present  and  wasting  time  with  time-outs  on  messages 
that  cannot  succeed. 

5.1  The  Problem 

Beyond  this,  however,  is  a  desire  for  more  graceful  degra¬ 
dation  in  behavior.  That  is,  failing  to  process  a  message 
because  of  the  inaccessibility  of  the  node  which  holds 
some  object  slot  (or  some  method,  in  the  remote  class 
case)  seems  severe.  In  many  cases,  it  may  be  that  the 
method  on  some  node  is  merely  a  fanciful  specializa¬ 
tion  of  some  inherited  method.  For  example,  the  say 
method  on  Student  may  just  print  something  silly  (like 
“Yo”)  then  proceed  to  invoke  the  parent  Person  say 
method.  In  such  a  situation,  it  might  be  desirable  to 
permit  the  student  behavior  to  gracefully  degrade  into 
normal  person  behavior  when  the  student  method  is  un¬ 
available.  Thus,  were  we  to  build  a  GradStudent  class 
that  delegates  to  Student  but  choose  not  to  export  the 
implementation  of  the  Student  say  method  to  the  grad 
student, then  attempts  to  make  a  grad  student  node 
say  something  would  be  fraught  with  peril  if  the  parent 
Student  node  were  crashed.  Even  when  a  grad  student 
forgets  for  a  moment  how  to  behave  like  a  student,  it 
would  be  nice  if  they  could  at  least  still  temporarily  act 
like  a  normal  person  in  the  meantime  rather  than  going 
totally  catatonic. 

Similarly,  the  node  at  which  some  slot  resides  may 
become  unavailable.  If  that  slot  shadows  some  other  slot 
with  the  same  name  but  later  in  the  delegation  chain, 
it  may  sometimes  be  desirable  to  permit  slot  reads  of 
the  unavailable  slot  to  be  deferred  as  slot  reads  of  the 
shadowed  slot.  Returning  to  Figure  1,  for  example,  if 
the  Student  nickname  slot  becomes  unavailable,  it  may 
be  acceptable  to  read  the  shadowed  Person  nickname 
slot  instead. 

We  have  attempted  to  deal  with  this  desirability  of 
graceful  inheritance  degradation  by  a  mechanism  we  call 
message  deferral  For  example,  we  would  say  that  when 
the  student  class  is  not  available,  it  may  be  safe  to  defer 
the  say  message  to  the  shadowed  person  class.  Some 
methods  may  be  safe  to  defer  while  others  may  not  be. 
For  example,  when  a  MobileObject  moves,  it  merely 
changes  its  place  slot  and  tells  the  place  object  to  up¬ 
date  its  inventory  to  include  the  new  object  (and  tell 
the  old  place  to  release  the  object).  For  Persons,  how¬ 
ever,  we  must  not  only  move  the  person  object  but  also 
move  each  of  the  objects  being  carried  in  the  person’s 

^^For  example,  like  when  GradStudent  behavioral  norms 
are  regulated  by  a  different  set  of  guidelines  than  typical 
Student  norms,  dude. 


possessions  slot.  Thus,  were  the  person  class  not  avail¬ 
able  to  some  student,  it  would  not  be  safe  to  defer  move 
messages  to  the  underlying  MobileObject  class  upon 
which  the  person  class  was  built.  To  do  so  would  mean 
to  lose  your  wallet  along  with  all  your  other  possessions! 

Similarly,  we  may  have  an  Employee  say  method  that 
is  specialized  to  say  “Sir”  then  go  on  to  call  the  Person 
say  method.  We  might  then  build  a  Manager  class  that 
delegates  to  Employee.  Were  we  to  then  ask  a  say  mes¬ 
sage  of  a  manager  whose  employee  node  is  unavailable, 
it  may  be  undesirable,  due  to  corporate  protocol,  to  de¬ 
fer  to  the  less  formal  person  say  method.  Similarly,  it 
might  be  embarrassing  to  defer  to  the  pedestrian  Person 
nickname  slot  when  an  attempt  is  made  to  access  the 
Employee  nickname  slot.  These  illustrate  a  subtle  com¬ 
plication  in  the  deferral  mechanism:  the  safety  of  a  de¬ 
ferral  is  not  just  a  property  of  the  message  or  slot  read 
being  deferred,  it  is  also  a  property  of  the  path  by  which 
the  deferral  takes  place.  In  our  GradStudent /Student 
say  the  deferral  to  Person  was  acceptable,  but  not  so  for 
the  Manager /Employee  deferral.  Thus,  when  attempting 
to  defer  a  message,  some  record  of  the  deferral  path  must 
be  passed  along  with  the  deferral  attempt.  Notice  also 
that  more  than  one  node  along  the  delegation  chain  may 
be  unavailable  at  one  time.  Thus,  an  arbitrary  depth  of 
deferral  may  become  necessary. 

Finally,  this  deferral  mechanism  is  further  compli¬ 
cated  by  multiple  inheritance.  Consider,  for  example, 
some  message  having  methods  along  both  the  primary 
parent  delegation  chain  and  the  secondary  parent  del¬ 
egation  chain.  If  one  chain  is  unavailable,  perhaps  the 
second  should  be  attempted.  In  such  a  situation,  the 
safety  and  propriety  of  deferral  may  involve  shadowing 
of  methods  not  via  child  shadowing  but  via  neighbor¬ 
ing  parent  shadowing.  Viewing  the  child/parent  paths 
as  vertically  directed  and  multiple  parents  of  a  node  as 
horizontally  splayed,  we  recognize  that  the  child/parent 
shadowing  is  a  matter  of  vertical  shadowing  while  multi¬ 
ple  parents  that  handle  the  same  method  could  be  said  to 
engage  in  horizontal  shadowing.  This  distinction  will  be¬ 
come  necessary  in  understanding  how  we  implement  safe 
deferral.  In  a  scenario  where  an  employee  is  a  student 
intern  (and  thus  inherits  from  both  a  normal  employee 
node  and  a  student  node),  this  issue  can  be  critical. 

5.2  The  Solution 

How  then  do  we  implement  this  selective  deferral  mech¬ 
anism?  Presently,  each  time  a  new  class  is  created  that 
delegates  to  a  parent  class,  the  parent  class  is  notified  of 
the  names  of  all  the  local  messages  that  the  child  class 
handles.  In  this  way,  if  the  parent  class  has  never  before 
been  delegated  to  by  a  child  of  the  new  child  class,  the 
parent  class  can  detect  which  of  its  methods  are  shad¬ 
owed  by  the  child.  This  same  list  of  the  child  class’s  local 
messages  is  then  forwarded  to  the  parent  class’s  parents’ 
classes,  and  so  on  down  the  class  delegation  chain,  so 
that  deeply  shadowed  messages  can  be  annotated.  Note 
that  at  each  point  where  a  shadowing  is  detected,  we 
store  in  the  shadowed  class  an  association  between  the 
shadowed  message  and  the  child  classes  which  shadow  it. 
If  we  choose  to  declare  a  particular  message  to  be  safe  to 


defer  to  some  specific  shadowed  parent,  we  can  specify 
that  in  the  class  definition  by  stating  precisely,  for  each 
local  method,  which  parent  class(es)  that  method’s  mes¬ 
sage  can  safely  be  deferred  to.  In  such  cases,  the  child’s 
message  shadowing  of  that  message  is  not  recorded  at 
the  parent  class. 

Note  also  that  this  shadow  information  is  propagated 
aggressively  at  the  earliest  possible  moment,  namely  at 
class  creation  time.  This  is  because  if  we  tried  to  imple¬ 
ment  a  lazy  on-demand  strategy,  by  the  time  we  actually 
need  to  attempt  a  message  deferral  it  may  well  be  too 
late  to  attempt  to  propagate  the  shadowing  information. 
Since  this  shadowing  information  is  our  means  for  judg¬ 
ing  if  a  particular  deferral  is  safe,  failing  to  propagate 
this  vital  information  could  be  our  undoing. 

To  see  how  this  cl2tss  shadowing  information  is  used 
to  detect  deferral  safety,  consider  the  following  scenario. 
An  object  receives  a  message  and  decomposes  it  into  a 
slot  access  on  some  ancestor  node.  If  this  ancestor  is 
unavailable,  it  may  be  desirable  to  attempt  to  defer  to 
some  other  ancestor  with  a  slot  of  the  same  name.  We 
therefore  consult  the  method  dispatch  table  for  an  ances¬ 
tor  whose  class  indicates  that  it  can  handle  the  message 
which  failed,  then  forward  the  message  on  to  the  ances¬ 
tor,  telling  it  that  this  is  a  deferral  and  telling  it  which 
ancestor (s)  we  have  deferred  across.  If  the  deferred  an¬ 
cestor  is  likewise  unavailable,  we  proceed  through  the 
ancestry  with  deeper  and  deeper  deferral  attempts.  Sim¬ 
ilarly,  if  the  attempt  to  discover  if  an  ancestor’s  class 
can  handle  this  message  involves  querying  a  remote  class 
which  is  likewise  unavailable  then  we  treat  that  ances¬ 
tor  as  unavailable.  If  we  run  out  of  deferral  candidates, 
then  we  finally  give  up  and  return  a  failure  signal.  On 
the  other  hand,  if  we  do  find  some  ancestor  available, 
then,  at  the  point  where  we  succeed,  the  receiving  an¬ 
cestor  processes  the  message  by  first  consulting  its  class’s 
association  of  shadowed  messages  and  the  child  classes 
which  shadow  them.  If  the  present  message  is  discovered 
to  be  shadowed  by  an  object  class  of  one  of  the  ances¬ 
tors  we  have  just  deferred  across,  then  the  deferral  is 
unsafe  and  we  send  an  error  to  that  effect.  Otherwise, 
we  proceed  with  the  message  as  usual. 

This  strategy  correctly  detects  vertical  shadowing,  as 
defined  above.  Unfortunately,  it  does  not  handle  hor¬ 
izontal  shadowing.  For  that,  we  need  additional  class 
information.  Specifically,  each  class,  upon  creation, 
queries  each  of  its  parent  classes  to  discover  the  full 
structure  of  messages  that  each  ancestral  class  handles. 
Now,  when  a  class  is  created  that  delegates  with  multi¬ 
ple  inheritance,  we  merely  arrange  that  the  list  of  child 
messages  that  the  parents  are  notified  of  includes  those 
messages  handled  by  neighboring  parents.  For  example, 
in  our  Person  class,  the  Animat eObject  class  would  be 
notified  of  the  local  messages  of  Person  as  well  as  all 
messages  of  MobileObject  (since  we  have  left-to-right 
precedence  in  our  multiple  inheritance).  When  we  then 
attempt  a  deferral  that  forces  us  to  try  the  secondary 

^^Even  with  opaque  remote  classes,  the  ancestral  mes¬ 
sages  are  revealed  since  this  does  not  expose  the  implemen¬ 
tation.  It  only  exposes  some  small  structural  property  of  the 
implementation. 


parent  Animat eObject  delegation  chain,  we  include  in 
our  trace  of  the  deferrals  we  have  already  attempted  all 
the  ancestors  tried  along  the  primary  parent  delegation 
chain.  In  this  way,  the  horizontal  shadowing  information 
is  given  to  the  delegates  that  may  be  asked  to  process 
a  deferred  message,  and  the  path  which  the  deferral  fol¬ 
lowed  includes  the  horizontally  neighboring  ancestors  of 
the  target  deferral  ancestor. 

In  closing,  note  that  we  have  expressed  this  book¬ 
keeping  of  vertical  and  horizontal  shadowings  in  terms 
of  messages  sent  among  classes.  We  did  so  because,  in 
general,  communication  with  remote  classes  may  require 
it.  In  the  case  where  a  local  class  must  communicate 
information  to  another  local  class,  the  message  send  is 
somewhat  fanciful.  Indeed,  with  some  compiler  exten¬ 
sions,  block  compiling  a  selection  of  classes  could  effect 
the  same  result  directly  in  the  internal  class  represen¬ 
tations.  We  have  not  explored  such  compiler  extensions 
since  we  wished  to  make  our  SchMUSE  implementation 
based  solely  on  portable  MIT  Scheme  language  features. 
Compiler  extensions  generally  do  not  port  quite  so  easily. 

6  Related  Work 

As  mentioned  at  the  outset,  our  delegation  style  ob¬ 
ject  system  was  modelled  after  [Adams  &  Rees  88].  Our 
advocacy  of  this  style’s  sharing  properties  and  sup¬ 
port  of  multiple  shared  views  comes  from  Clovers 
[Stein  &  Zdonik  89],  although  our  introduction  of  classes 
into  the  delegation  style  is  not  traditional  in  the  sense 
that  it  is  not  prototype-based  delegation  [Lieberman  86]. 

Of  the  distributed  object  systems  we  have  examined, 
nearly  all  have  opted  to  extend  a  traditional  class-based 
inheritance  system  to  a  distributed  object  system  only  to 
afterward  comment  that  a  delegation-based  object  sys¬ 
tem  would  have  been  more  desirable,  as  was  anticipated 
by  [Otten  &  Hagen  90].  Most  of  them  cite  the  extra  fiex- 
ibility  provided  by  delegation  systems  as  the  motivating 
factor.  [Bennett  90]  even  goes  to  the  extent  of  identify¬ 
ing  remote  classes  as  the  primary  point  of  tension,  argu¬ 
ing  that  delegation  systems,  by  normally  storing  meth¬ 
ods  along  with  the  local  slots  inside  objects,  naturally 
satisfy  this  key  flexibility  concern.  Their  careful  exami¬ 
nation  of  the  design  options  surrounding  remote  classes 
was  very  helpful  in  justifying  to  ourselves  our  constraint 
that  local  object  nodes  must  have  local  class  information 
about  the  node’s  class.  Unlike  them,  however,  we  opted 
to  promote  classes  to  the  status  of  objects  themselves, 
complete  with  their  own  kind  of  class  reference  (namely, 
ClassTickets).  We  find  our  solution  to  this  problem 
of  remote  classes,  therefore,  to  be  fairly  elegant  with¬ 
out  requiring  delegation-based  object  instances  to  store 
their  methods  directly  in  their  representations.  Rather, 
we  use  class  information  to  establish  a  method  dispatch 
table  separate  from  the  object  slots  array. 

Our  use  of  globally  unique  tickets  was  inspired  by 
[Leach  et  al.  82].  It  differs  from  the  apparently  com¬ 
mon  use  of  object  ‘‘proxies”  for  remote  object  references 
[Decouchant  86].  Proxies  are  objects  that  accept  mes¬ 
sages  and  forward  them  to  a  remote  object.  We  instead 
make  our  message  send  procedure  detect  if  the  object 


being  queried  is  local  or  remote  (which  is  easily  deter¬ 
mined  by  inspecting  the  object  ticket’s  realm  ticket)  and 
remotely  forward  the  message  if  it  is  remote.  It  seems 
to  us  that  the  use  of  proxies  is  an  apology  for  having 
an  awkward  object  reference  mechanism  or  for  having 
a  system  which  demands  every  target  of  a  message  be 
an  “object”  in  some  built-in  language  specific  sense.  In 
fact,  [McCullough  87]  even  goes  so  far  as  to  then  pro¬ 
vide  global  UIDs  for  their  proxies,  along  with  a  table  to 
map  firom  proxies  to  UIDs  to  test  object  identity.  This 
seems  like  a  further  apology  for  having  used  proxies  at 
all.  Since  we  were  not  retro-fitting  our  remote  object 
references  to  an  existing  local  object  language  system, 
we  were  not  compelled  to  make  such  apologies. 

Some 

distributed  object  systems  [Nascimento  &  Dollimore  92] 
[Feeley  &  Levy  92]  have  adopted  version  numbers  for 
dealing  with  object  and  class  references.  A  similar  com¬ 
mon  approach  is  to  adopt  a  centralized  authority  to  co¬ 
ordinate  the  issuance  of  sequential  object  ID  numbers. 
Any  such  strategies  must  globally  synchronize  their  num¬ 
bering  to  some  extent  to  avoid  collision.  Our  globally 
unique  identifiers  require  no  such  synchronization.  This 
makes  our  system  scalable.  To  wit,  our  strategy  could 
support  a  SchMUSE  server  at  every  single  Internet 
address.  Moreover,  although  we  shall  permit  “pointer 
snapping”  of  RealmTickets  within  object/class  tickets 
when  those  entities  migrate  among  realms,  the  remain¬ 
der  of  the  globally  unique  ID  remains  stable,  as  an  im¬ 
mutable  birthtime/birthplace  of  each  object.  This  has 
greatly  enhanced  debugging  and  profiling  of  our  system. 

Our  realm  resolution  is  currently  loosely  modelled  af¬ 
ter  Internet  namespace  service.  We  intend  to  investigate 
alternative  distributed  resolution  strategies,  such  as  the 
“clearinghouse”  approach  of  [Oppen  &  Dalai  83].  Ad¬ 
mittedly,  given  the  small  size  of  our  prototype  lab  net¬ 
work  (48  workstations;  2  file  servers),  we  have  not  been 
motivated  to  explore  fanciful  variations.  Now  that  MIT 
has  completed  its  installation  of  Internet  drops  within 
campus  dorm  rooms,  the  motivation  to  explore  creative 
options  is  expected  to  grow  and  we  anticipate  a  veritable 
army  of  participants  avid  to  explore  those  options. 

Finally,  our  blurring  of  the  distinction  between  client 
and  server  in  the  SchMUSE  is  similar  to  that  found  in 
Flamingo  [Anderson  86].  Their  system,  however,  is  a 
cooperative  mailer  system;  ours  is  an  interactive  simula¬ 
tion  environment,  so  the  types  of  client/server  interac¬ 
tions  are  characteristically  different. 

7  Contributions  and  Conclusions 

Our  novel  use  of  globally  unique  tickets  as  object  ref¬ 
erences  has  proven  to  be  elegant,  versatile,  low-cost, 
and  scalable  without  requiring  network-wide  clock  syn¬ 
chronization.  Further,  our  separation  of  object  identi¬ 
fication  from  object  location,  by  means  of  embedding 
RealmTickets  within  object  and  class  tickets,  has  proven 
quite  useful.  It  allows  course-grained  object  migration 
(by  realm)  without  requiring  forwarding  addressing  to 
disrupt  the  object  database.  This  was  achieved  by  us¬ 
ing  a  separate  realm  table  to  map  the  realm  tickets  to 


actual  machine  locations.  This  mapping  has  been  proto¬ 
typed  as  a  “Brazilian”  distributed  hierarchical  lazy  map¬ 
ping  strategy  modelled  after  Internet  namespace  servers. 
Again,  this  will  scale  at  least  as  well  as  the  Internet  has 
scaled. 

We  believe  that  our  most  significant  contribution  is 
our  incorporation  of  class  information  into  a  delegation- 
based  inheritance  system,  yielding  what  we  call  a  class- 
based  delegation  system.  We  know  of  no  other  distributed 
object  system  that  has  wed  these  traditionally  rivaled 
approaches  of  delegation  and  class  inheritance. 

Moreover,  we  found  that  class-based  delegation  facili¬ 
tates  several  novel  issues  arising  from  remote  delegation 
which  have  heretofore  been  unaddressed  by  the  object- 
oriented  community.  For  example,  we  are  unaware 
of  previous  work  in  accelerating  delegation  through 
compile-time  class  analysis  in  a  setting  where  classes 
themselves  may  be  remote  (and  hence,  compile-time 
opaque).  In  addition,  we  have  also  introduced  a  novel 
notion  of  message  deferral  which  permits  graceful  degra¬ 
dation  of  object  behavior  and  system  availability  as  re¬ 
mote  sites  go  down  while  paying  heed  to  the  safety  and 
propriety  of  certain  deferral. 

8  Future  Work 

There  are  many  directions  for  further  development  of 
MIT  SchMUSE.  Roughly,  these  include  object  migra¬ 
tion,  enhanced  concurrency,  access  control,  and  commu¬ 
nication  security. 

Foremost  on  our  agenda  is  investigating  mechanisms 
for  allowing  objects  to  migrate  from  one  realm  to  an¬ 
other.  We  will  likely  follow  the  fine  work  of  the  Emer¬ 
ald  Project  [Jul  et  al.  88]  [Jul  88],  including  their  for¬ 
warding  address  strategy  [Fowler  86]  [Fowler  85]  (for 
“snapping”  RealmTickets  in  stale  ObjectTickets  to 
point  to  the  new  realm  where  the  object  has  moved) 
as  well  as  their  distributed  garbage  collection  algo¬ 
rithm  [Vestal  87]  (to  GO  stale  forwarding  addresses).  Of 
course,  distributed  GC  is  valuable  for  automatic  stor¬ 
age  reclamation  in  general  anyway  so  that  space  quo¬ 
tas  with  manual  object  de-allocation  do  not  have  to  be 
imposed  on  users.  Although  the  above  citations  give 
the  impression  that  each  of  these  extensions  is  a  solved 
problem,  actually  implementing  them  should  prove  a  re¬ 
spectable  engineering  task.  More  recent  advances  in  dis¬ 
tributed  GC  should  prove  helpful  [Detlefs  90]  [VAT  92] 
[Maheshwari  93]. 

Also,  we  would  like  to  support  a  higher  degree  of  con¬ 
currency  in  the  SchMUSE  network  by  providing  a  true 
transaction-based  mechanism  for  our  message  passing. 
(Currently,  our  messages  are  not  even  atomic  in  that  par¬ 
tially  completed  messages  which  fail  do  not  back  out). 
We  will  likely  pursue  a  design  based  on  nested  trans¬ 
actions  [Moss  81],  possibly  with  some  degree  of  opti¬ 
mistic  concurrency  among  concurrent  transactions.  This 
should  prove  a  fairly  challenging  task. 

Next,  the  present  prototype  of  the  SchMUSE  pro¬ 
vides  a  fairly  exotic  locking  strategy  for  object  access 
control.  This  is  documented  in  [Cohen  93].  At  present, 
however,  it  provides  only  fine-  and  coarse-grained  lock¬ 
ing  (namely,  instance  locking  per  message  and  instance 


locking  per  instance  creator).  We  would  like  to  extend  it 
to  provide  medium-grained  locking  as  well  (that  is,  lock¬ 
ing  per  instance).  This  is  a  straightforward  extension. 

Our  initial  implementation  uses  [Adams  &  Rees  88] 
style  method  delegation.  We  intend  to  look  into  a 
more  DvLAN-like  generic  function  method  specialization 
mechanism  for  overriding  default  class  methods. 

Finally,  it  would  also  be  interesting  to  support  net¬ 
work  privacy  by  encrypting  messages  and  their  results  as 
they  are  shipped  between  SchMUSE  sites.  This  should 
prove  a  fairly  painless  extension. 
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